A reconfigurable frame interpolation hardware architecture for high definition video

Since Frame Rate Up-Conversion (FRC) is started to be used in recent consumer electronics products like High Definition TV, real-time and low cost implementation of FRC algorithms has become very important. Therefore, in this paper, we propose a low cost hardware architecture for realtime implementation of frame interpolation algorithms. The proposed hardware architecture is reconfigurable and it allows adaptive selection of frame interpolation algorithms for each Macroblock. The proposed hardware architecture is implemented in VHDL and mapped to a low cost Xilinx XC3SD1800A-4 FPGA device. The implementation results show that the proposed hardware can run at 101 MHz on this FPGA and consumes 32 BRAMs and 15384 slices

A High performance and low cost hardware arcitecture for H.264 transform and quantization algorithms

In this paper, we present a high performance and low cost hardware architecture for real-time implementation of forward transform and quantization and inverse transform and quantization algorithms used in H.264 / MPEG4 Part 10 video coding standard. The hard-ware architecture is based on a reconfigurable datapath with only one multiplier. This hardware is designed to be used as part of a complete low power H.264 video coding system for portable appli-cations. The proposed architecture is implemented in Verilog HDL. The Verilog RTL code is verified to work at 81 MHz in a Xilinx Virtex II FPGA and it is verified to work at 210 MHz in a 0.18´ ASIC implementation. The FPGA and ASIC implementations can code 27 and 70 VGA frames (640x480) per second respectively

H.264 Intra frame coder system design /

Recently, a new international standard for video compression named H.264 / MPEG4 Part 10 is developed. This new standard offers significantly better video compression efficiency than previous International standards. However, this coding gain comes with an increase in encoding complexity. This makes it impossible to implement a real time H.264 video coder using the state-of-the-art embedded processor alone. Therefore, in this thesis, we developed an FPGA-based H.264 intra frame coder system for portable applications targeting level 2.0 of baseline profile. As part of the system, we first designe a high performance and low cost hardware architecture for real-time implementation of forward transform and quantization and inverse transform and quantization algorithms used in H.264 / MPEG4 Part 10 video coding standard in Verilog HDL. The design is first veryfied with RTL simulations using Mentor Graphics Modelism. It is then verified to work on a Xilinx Virtex II FPGA on an ARM Versatile Platform development board. We then designed the top-level H.264 Intra Frame Coder System targeting 30 fps CIF encoding. The system consists of search, mode decision and coding parts. The mode decision part implements a Hadamard Transform based mode decision algorithm. The coding part is implemented by integrating Transform-Quant module with CAVLC and Intra Prediction modules. The top-level design is verified with RTL simulations using Menthor Graphics Modelism. The complete H.264 Intra Frame Coder System is verified to work on an ARM Versatile Platform development board. The verification includes first capturing an RGB image, converting it into YCbCr format, partitioning the image into macroblocks, and writing it into a SRAM using the software running on ARM9EJ-S processor. Then the intra frame coder hardware mapped to the Xilinx Virtex II FPGA using Leonardo Spectrum and Xilinx ISE is used to encode the image and reconstruct it. The conversion of reconstucted image into raster scan order and RGB color domain is then performed by software running on ARM9EJ-S processor. The reconsructed image is then displayed on acolor LCD panel for visual verification

Motion estimation based frame rate conversion hardware designs

Frame Rate Up-Conversion (FRC) is the conversion of a lower frame rate video signal to a higher frame rate video signal. FRC algorithms using Motion Estimation (ME) obtain better quality results. Among the block matching ME algorithms, Full Search (FS) achieves the best performance since it searches all search locations in a given search range. However, its computational complexity, especially for the recently available High Definition (HD) video formats, is very high. Therefore, in this thesis, we proposed new ME algorithms for real-time processing of HD video and designed efficient hardware architectures for implementing these ME algorithms. These algorithms perform very close to FS by searching much fewer search locations than FS algorithm. We implemented the proposed hardware architectures in VHDL and mapped them to a Xilinx FPGA. ME for FRC requires finding the true motion among consecutive frames. In order to find the true motion, Vector Median Filter (VMF) is used to smooth the motion vector field obtained by block matching ME. However, VMFs are difficult to implement in real-time due to their high computational complexity. Therefore, in this thesis, we proposed several techniques to reduce the computational complexity of VMFs by using data reuse methodology and by exploiting the spatial correlations in the vector field. In addition, we designed an efficient VMF hardware including the computation reduction techniques exploiting the spatial correlations in the motion vector field. We implemented the proposed hardware architecture in Verilog and mapped it to a Xilinx FPGA. ME based FRC requires interpolation of frames using the motion vectors found by ME. Frame interpolation algorithms also have high computational complexity. Therefore, in this thesis, we proposed a low cost hardware architecture for real-time implementation of frame interpolation algorithms. The proposed hardware architecture is reconfigurable and it allows adaptive selection of frame interpolation algorithms for each Macroblock. We implemented the proposed hardware architecture in VHDL and mapped it to a low cost Xilinx FPGA

Dynamically variable step search motion estimation algorithm and a dynamically reconfigurable hardware for its implementation

Motion Estimation (ME) is the most computationally intensive part of video compression and video enhancement systems. For the recently available High Definition (HD) video formats, the computational complexity of De full search (FS) ME algorithm is prohibitively high, whereas the PSNR obtained by fast search ME algorithms is low. Therefore, ill this paper, we present Dynamically Variable Step Search (DVSS) ME algorithm for Processing high definition video formats and a dynamically reconfigurable hardware efficiently implementing DVSS algorithm. The architecture for efficiently implementing DVSS algorithm. The simulation results showed that DVSS algorithm performs very close to FS algorithm by searching much fewer search locations than FS algorithm and it outperforms successful past search ME algorithms by searching more search locations than these algorithms. The proposed hardware is implemented in VHDL and is capable, of processing high definition video formats in real time. Therefore, it can be used in consumer electronics products for video compression, frame rate up-conversion and de-interlacing(1)

Nonparametric joint shape and feature priors for segmentation of dendritic spines

Multimodal shape density estimation is a challenging task in many biomedical image segmentation problems. Existing techniques in the literature estimate the underlying shape distribution by extending Parzen density estimator to the space of shapes. Such density estimates are only expressed in terms of distances between shapes which may not be sufficient for ensuring accurate segmentation when the observed intensities provide very little information about the object boundaries. In such scenarios, employing additional shape-dependent discriminative features as priors and exploiting both shape and feature priors can aid to the segmentation process. In this paper, we propose a segmentation algorithm that uses nonparametric joint shape and feature priors using Parzen density estimator. The joint prior density estimate is expressed in terms of distances between shapes and distances between features. We incorporate the learned joint shape and feature prior distribution into a maximum a posteriori estimation framework for segmentation. The resulting optimization problem is solved using active contours. We present experimental results on dendritic spine segmentation in 2-photon microscopy images which involve a multimodal shape density

On comparison of manifold learning techniques for dendritic spine classification

Dendritic spines are one of the key functional components of neurons. Their morphological changes are correlated with neuronal activity. Neuroscientists study spine shape variations to understand their relation with neuronal activity. Currently this analysis performed manually, the availability of reliable automated tools would assist neuroscientists and accelerate this research. Previously, morphological features based spine analysis has been performed and reported in the literature. In this paper, we explore the idea of using and comparing manifold learning techniques for classifying spine shapes. We start with automatically segmented data and construct our feature vector by stacking and concatenating the columns of images. Further, we apply unsupervised manifold learning algorithms and compare their performance in the context of dendritic spine classification. We achieved 85.95% accuracy on a dataset of 242 automatically segmented mushroom and stubby spines. We also observed that ISOMAP implicitly computes prominent features suitable for classification purposes

A joint classification and segmentation approach for dendritic spine segmentation in 2-photon microscopy images

Shape priors have been successfully used in challenging biomedical imaging problems. However when the shape distribution involves multiple shape classes, leading to a multimodal shape density, effective use of shape priors in segmentation becomes more challenging. In such scenarios, knowing the class of the shape can aid the segmentation process, which is of course unknown a priori. In this paper, we propose a joint classification and segmentation approach for dendritic spine segmentation which infers the class of the spine during segmentation and adapts the remaining segmentation process accordingly. We evaluate our proposed approach on 2-photon microscopy images containing dendritic spines and compare its performance quantitatively to an existing approach based on nonparametric shape priors. Both visual and quantitative results demonstrate the effectiveness of our approach in dendritic spine segmentation

Dendritic spine shape classification from two-photon microscopy images (Dendritik diken şekillerinin iki foton mikroskopi görüntüleri kullanılarak sınıflandırılması)

Functional properties of a neuron are coupled with its morphology, particularly the morphology of dendritic spines. Spine volume has been used as the primary morphological parameter in order the characterize the structure and function coupling. However, this reductionist approach neglects the rich shape repertoire of dendritic spines. First step to incorporate spine shape information into functional coupling is classifying main spine shapes that were proposed in the literature. Due to the lack of reliable and fully automatic tools to analyze the morphology of the spines, such analysis is often performed manually, which is a laborious and time intensive task and prone to subjectivity. In this paper we present an automated approach to extract features using basic image processing techniques, and classify spines into mushroom or stubby by applying machine learning algorithms. Out of 50 manually segmented mushroom and stubby spines, Support Vector Machine was able to classify 98% of the spines correctly